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Opinions ,  interpretations,  conclusions  and  recommendations  are 
those  of  the  author  and  are  not  necessarily  endorsed  by  the  U.S. 
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x  Where  copyrighted  material  is  quoted,  permission  has  been 
obtained  to  use  such  material . 

x  Where  material  from  documents  designated  for  limited 
distribution  is  quoted,  permission  has  been  obtained  to  use  the 
material . 

x  Citations  of  commercial  organizations  and  trade  names  in  this 
report  do  not  constitute  an  official  Department  of  Army 
endorsement  or  approval  of  the  products  or  services  of  these 
organizations . 
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5.  INTRODUCTION 


Based  on  preliminary  evidence  from  combined  change  detection  and  neural  network  classifier  in 
our  CAD  research,  the  purpose  of  the  project  is  to  develop  an  automatic  change  detection  method  to 
quantitatively  extract  the  clinically  important  changes  of  suspicious  lesions,  upgrade  the  existing  CAD 
system,  and  thus  improve  the  clinical  diagnosis  of  breast  cancer.  We  will  build  a  site  model  for  each 
individual  patient  for  monitoring  the  breast  tissue  changes  and  extend  our  current  research  on  image 
registration,  soft  tissue  modeling,  and  image  segmentation,  to  the  early  detection  of  breast  cancer. 
Specific  aims  include:  1)  registration  and  segmentation  of  deformable  breast  tissue  structures  across  a 
series  of  mammograms;  2)  construction  of  a  site  model  of  the  mammogram  for  individual  patients 
showing  the  locations  of  regions  of  interest  and  associated  diagnostic  information;  3)  identification  of 
clinically  significant  changes  in  both  global  and  local  mass  areas  within  the  breast;  and  4)  integration  and 
evaluation  of  the  developed  techniques  with  existing  CAD  prototype.  At  conclusion  of  this  project,  we 
anticipate  achieving  the  following:  1)  establish  a  reliable  technique  of  monitoring  breast  tissue  changes 
associated  with  cancerous  masses;  2)  deliver  a  CAD  prototype  that  can  incorporate  tissue  change 
information  from  additional  mammograms;  3)  evaluate  the  merit  of  combining  change  detection  and  CAD 
for  improved  clinical  diagnosis  using  multiple  mammograms;  and  4)  acquire  the  experience  necessary  to 
explore  multimodality  imaging  for  unified  detection,  diagnosis  and  treatment  assessment  of  breast  cancer. 
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6.-  BODY-Annual  Summary 

The  long-term  goal  of  this  career  development  project  is  to  develop  image  guided  diagnosis 
methodology  through  change  detection  in  multimodality  image  sequences  for  breast  cancer.  The  research 
requires  the  knowledge  of  image  analysis,  image  registration,  computer  graphics,  and  information  theory. 


As  the  training  accomplishments  during  the  first  year, 

I  have  developed  a  close  consultation  relationship  with  Dr. 

Matthew  Freedman  (radiologist)  and  Dr.  Ben  Lo  (medical 
physicist)  at  Georgetown  University  Medical  Center.  I  have 
also  developed  a  strategic  collaboration  with  Dr.  Robert 
Clarke  (Department  of  Oncology)  at  the  Lombardi  Cancer 
Center.  Through  them,  I  have  learned  more  about  breast 
cancer  at  both  imaging  and  molecular  levels.  I  am  now  an 
invited  member  attending  the  weekly  meeting  organized  by 
Dr.  Freedman  and  Dr.  Clarke. 

Based  on  this,  I  have  also  developed  my  own  research 
group,  as  the  lab  director,  with  one  research  associate 
professor  and  eight  graduate  students.  I  am  currently  serving 
as  the  major  advisor  for  three  graduate  students  who  are 
specifically  working  on  the  breast  cancer  research.  I  also  served  as  a  penal  member  of  the  Peer-Review 
Committee  for  Department  of  Defense  Breast  Cancer  Research  Program  in  1999. 1  have  been  serving  as  a 
member  of  the  Technical  Committee  (TC)  on  Neural  Networks  for  Signal  Processing  (NNSP)  in  IEEE 
since  1999. 

As  the  research  accomplishments  during  the  first  year, 

I  have  first  identified  the  following  tasks  as  the  first  step  of 
the  project: 

1 .  Analyze  medical  image  (both  mammograms  and  magnetic 
resonance  image)  that  contains  multiple  anatomical 
objects  through  direct  2-D  and/or  3-D  tissue  quantification 
and  region  segmentation.  In  particular,  the  tissue 
quantification  is  performed  based  on  the  standard  finite 
normal  mixture  (SFNM)  modeling  of  pixel  image 
distribution,  AIC  and  MDL  guided  model  selection,  and 
EM  maximum  likelihood  model  estimation.  The  region 
segmentation  (e.g.,  the  mass  sites  from  the  mammograms) 
is  achieved  based  on  inhomogeneous  MRF  modeling  of 
context  images  and  relaxation  labeling  of  pixel 
memberships. 

2.  Construct  a  patient  specific  model  (i.e.,  the  site  model) 
based  on  the  outcome  of  image  analysis  including  objects, 
surface,  and  boundaries,  of  the  normal  tissues  and 
detected/suspected  lesions.  This  will  provide  a  framework 
for  (1)  high  accuracy  change  monitoring  considering  the 
patient  variation  and  (2)  effective  data  fusion 
incorporating  prior/domain  specific  information. 

3.  Develop  a  multiple  step  algorithm  for  2-D  and  3-D  image 
registration  of  image  sequence  data  sets  and 
multimodality  image  data  sets.  It  consists  of  three  major 
components:  (1)  principle  axes  registration  (PAR),  (2)  site 
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Figure  2.  Mammogram  segmentation  (Up:  global). 
Multiple  level  MR  image  segmentation  (Middle). 
Lesion  site  segmentation  (Down:  local). 


.  model  support  control  feature  alignment  with  localized  PAR,  and  (3)  raw  data  matching  via 


maximizing  the  mutual  information  (MI)  between  the  two  images. 


4.  Apply  new  algorithms  to 
perform  change  detection  from  a 
set  of  sequence  images,  where 
the  clinical  objectives  are  lesion 
verification/detection,  lesion 
localization,  and  change 
quantification. 

Follow  this  plan,  major  research 
accomplishments  include: 


6.1  Lesion  site  segmentation 


We  have  developed  a  new  algorithm  to  perform  lesion  site  segmentation  as  well  as  whole  image 


segmentation.  We  have  implemented  the 
computer  codes  and  pilot  tested  its  effective 
applications  to  the  digital  phantoms, 
mammograms,  MR  images,  and  ultrasound 
images.  The  algorithm  includes  dual 
morphological  filtering  for  signal  enhancement 
and  statistical  model  based  tissue  quantification 
and  lesion  segmentation  [1,2].  Figure  1  shows 
the  results  of  segmentation  algorithm  with 
simulated  digital  phantom.  Figure  2  shows  the 
results  of  the  applications  to  multimodality 
images.  Our  results  have  indicated  that  all  the 
suspected  lesion  sites  were  successfully  detected 
and  the  areas  were  accurately  segmented. 
Detailed  discussions  please  see  the  attached 
manuscripts. 


6.2  Construction  of  Datient  specific  site  model  for  change  detection 


Based  image  analysis  results  and  all 
available  clinical  diagnostic  information,  we  pilot 
constructed  a  patient  specific  site  model.  This 
model  is  a  mathematical  formulation  of  multimedia 
scene  information,  mainly  including  object 
geometry  (e.g.,  object  location),  reference  labels 
(e.g.,  control  points  and/or  objects),  and  expert’s 
knowledge  (e.g.,  lesion  index,  diagnostic 
comments,  etc.).  Figure  3  shows  the  schematic 
flowchart  of  the  approach.  Figure  4  illustrates  the 
partial  results  of  the  site  models. 

6.3  Development  of  multiple-step  image 


registration  algorithm 


To  achieve  accurate  change  detection  for 
the  diagnosis  of  early  breast  cancer,  image 
registration  of  the  image  sequence  of  the  same  patient  is  a  crucial  step.  This  step  is  being  achieved 
through  multi-object  principal-axes  alignment,  site  model  supported  skeleton  mapping,  and  surface  based 


deformable  warping.  We  have  we  theoretically  verified  the  optimality  of  the  method  in  the  cases  of 
lacking  in  point-pair  correspondence,  by  considering  both  image  registration  can  be  divided  into  two 
categories:  (1)  object-imager  positioning  and  (2)  object-object 
mapping. 

Let  an  image  be  denoted  by  a  function  F(x,y,z)  where  F  is 
the  gray-level  of  the  pixel.  The  corresponding  graphics  is  denoted 
by  a  set  {(x,y,z)}  where  (x,y,z)  is  the  point  based  on  which  the 
graphics  is  formed.  Graphics  that  may  be  considered  as  a 
representation  of  the  image  through  a  pathway  of  segmentation, 
boundary  formation,  surface  representation,  or  feature  extraction. 

A  site  model  may  be  constructed  by  integrating  F(x,y,z)  and 
{(x,y,z)}  to  describe  a  particular  site.  Thus,  image  registration 
should  include  the  following  components:  assumptions,  criterion,  principle,  and  transformation.  Our 
research  has  indicated  that  the  major  problems  in  most  existing  image  registration  algorithms  are:  1) 
difficult  in  unifying  these  four  components; 

2)  difficult  in  proving  the  optimality  of  a 
particular  method;  and  3)  difficult  in 
resolving  the  conflict  between  registration 
and  "change". 

We  have  implemented  a  global 
principal-axes  registration  (GPAR) 
algorithm.  Its  operation  is  based  on  a  set  of 
"control  points/objects"  extracted/identified 
from  the  images.  Since  the  set  of  control 
points/objects  is  selected,  it  can  be  highly 
insensitive  to  the  "change".  When  point 
correspondence  is  available  and  least  square 
error  is  used  as  the  criterion  function,  it  has 
been  shown  that  the  solution  can  be 
obtained  through  a  point  set  matching,  where  the  similarity  transformation  is  derived  directly  from  a 
cross-covariance  matrix.  This  approach  may  not  be  applicable  to  control  object  matching  if  the  point 
correspondence  is  unavailable.  However,  if 
a  point-based  skeleton  matching  is  pursued, 
this  method  may  be  applicable.  In  addition, 
it  has  not  been  shown  that  this  approach 
works  for  "scaling"  operation  though  it  is 
highly  possible.  When  point 
correspondence  is  unavailable  and  least 
relative  entropy  is  used  as  the  criterion 
function,  it  can  be  shown  that  the  solution  is 
obtained  through  a  probability  density 
function  matching,  where  the  similarity 
transformation  is  derived  from  the  two  auto-covariance  matrices.  Unfortunately,  there  are  two  problems 
associate  with  the  present  approach.  First,  the  "rotation"  operation  is  defined  by  the  similarity 
transformation  matrix,  in  which  the  eigen-vectors  corresponds  to  the  principal  axes  of  the  sets.  However, 
although  the  "orientation"  of  each  of  the  principal  axes  is  uniquely  determined,  it  can  have  two  opposite 
"directions"  reflected  by  the  sign  of  the  corresponding  eigen-value.  This  event  is  called  "reflection"  (i.e., 
rotation  with  p+q  instead  of  q)  and  can  be  corrected  by  the  method  proposed  by  T.  S.  Huang.  Second, 
pseudo-symmetry  of  the  control  point/object  set  will  cause  a  large  intrinsic  error  in  the  "rotation" 
operation,  where  the  order  of  the  eigen-values  may  be  incorrectly  determined.  This  problem  may  be 
avoided  by  a  careful  selection  of  the  control  point/objects. 
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We  have  put  considerable  effort  to  guide  the  control  object  selection.  Our  experience  indicate  that 
the  control  points/objects  should  be  selected  according  to  the 
following  criteria:  1)  A  group  of  control  objects  with  each  of  the 
objects  having  the  same  order  of  the  points  (via  interpolation  or 
down-sampling;  2)  The  global  geometric  configuration  of  the 
control  objects,  i.e.,  the  spatial  distribution,  should  form  a  unique 
and  non-symmetry  pattern;  3)  Each  of  the  control  objects  prefers 
its  unique  and  non-symmetry  shape;  4)  The  local  geometric  shape 
of  the  control  objects  allows  small  distortions;  and  5)  The  global 
geometric  configuration  of  the  control  objects  assumes  shift- 
invariant.  In  addition,  a  patient  site  model  should  contain  a  set  of 
control  points/objects,  though  the  site  model  may  contain  much 
more  information  using  different  media. 

6.4  Integration  of  change  detection  with  CAD 

We  have  pilot  integrated  our  preliminary  change  detection 
capability  with  the  existing  CAD  system.  The  purpose  of  change 
detection  is  twofold:  (1)  to  detection  suspected  lesion  sites  as  the 
candidate  “input”  to  the  CAD  system,  and  (2)  to  monitor  the 
changes  of  the  “output”  lesion  sites  selected  by  the  CAD  system 
for  follow-up  diagnosis.  This  is  the  major  objective  in  the 
following  year  research.  However,  fruitful  preliminary  results  can 
be  found  in  our  attached  manuscripts  [1,2,3]. 

In  summary,  we  have  developed  a  statistical  model  supported  approach  for  enhanced  segmentation 
and  extraction  of  suspicious  mass  areas  from  mammographic  images.  With  an  appropriate  statistical 
description  of  various  discriminate  characteristics  of  both  true  and  false  candidates  from  the  localized 
areas,  an  improved  mass  detection  may  be  achieved  in  computer-aided  diagnosis.  In  this  study,  one  type 
of  morphological  operation  is 
derived  to  enhance  disease 
patterns  of  suspected  masses 
by  cleaning  up  unrelated 
background  clutters,  and  a 
model-based  image 

segmentation  is  performed  to 
localize  the  suspected  mass 
areas  using  stochastic 
relaxation  labeling  scheme. 

We  discuss  the  importance  of 
model  selection  when  a  finite 
generalized  Gaussian  mixture 
is  employed,  and  use  the 
information  theoretic  criteria  to  determine  the  optimal  model  structure  and  parameters.  Examples  are 
presented  to  show  the  effectiveness  of  the  proposed  methods  on  mass  lesion  enhancement  and 
segmentation  when  applied  to  mammographical  images.  Experimental  results  demonstrate  that  the 
proposed  method  achieves  a  very  satisfactory  performance  as  a  pre-processing  procedure  for  mass 
detection  in  computer-aided  diagnosis. 

Based  on  the  enhanced  segmentation  of  suspicious  mass  areas,  further  development  of  computer 
aided  mass  detection  may  be  decomposed  into  three  distinctive  machine  learning  tasks:  (1)  construction  of 
the  featured  knowledge  database;  (2)  mapping  of  the  classified  and/or  unclassified  data  points  in  the 
database;  and  (3)  development  of  an  intelligent  user  interface.  A  decision  support  system  may  then  be 
constructed  as  a  complementary  machine  observer  that  should  enhance  the  radiologists  performance  in 
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mass  detection.  We  adopt  a  mathematical  feature  extraction  procedure  to  construct  the  featured 
knowledge  database  from  all  the  suspicious  mass  sites  localized  by  the  enhanced  segmentation.  The 
optimal  mapping  of  the  data  points  is  then  obtained  by  learning  the  generalized  normal  mixtures  and 
decision  boundaries,  where  a  probabilistic  modular  neural  network  is  developed  to  carry  out  both  soft  and 
hard  clustering.  A  visual  explanation  of  the  decision  making  is  further  invented  as  a  clinical  support, 
based  on  an  interactive  visualization  hierarchy  through  the  probabilistic  principal  component  projections 
of  the  knowledge  database  and  the  localized  optimal  displays  of  the  retrieved  raw  data.  A  prototype 
system  is  developed  and  pilot  tested  to  demonstrate  the  applicability  of  this  framework  to  mammographic 
mass  detection. 


7;  APPENDICES 


7.1  Key  Research  Accomplishments 

•  We  are  creating  a  unique  mammography  database  consisting  of  a  sequence  of  mammograms  of  the 
same  patient  taken  over  a  period  of  time.  This  provides  an  opportunity  of  tracking  various  changes  of 
the  suspected  mass  lesions.  In  addition,  we  have  started  to  acquire  the  corresponding  three- 
dimensional  (3-D)  magnetic  resonance  (MR)  images  of  the  same  patient  with  the  purpose  of 
validating  the  computer  analysis  results  from  the  mammograms  using  the  3-D  MR  images. 

•  A  morphological  mass  signal  enhancement  method  is  developed  that  has  shown  its  effectiveness  on 
achieving  an  improved  lesion  localization  and  site  area  segmentation.  In  addition,  a  statistical  model- 
based  segmentation  algorithm  is  implemented  to  accurately  extract  mass-like  site  areas.  One 
manuscript  on  this  topic  has  been  submitted  to  IEEE  Transactions  on  Medical  Imaging. 

•  We  have  examined  a  total  of  nine  features  associated  with  mass-like  lesion  description.  We  have 
developed  a  visual  database  mapping  technique  to  interpret  the  machine  learning  and  decision  making. 
Our  preliminary  experiment  has  shown  that  three  of  the  nine  features  form  a  good  differentiation 
vector.  One  manuscript  on  this  topic  has  been  submitted  to  IEEE  Transactions  on  Medical  Imaging. 

•  We  have  proposed  a  hybrid  algorithm  for  2-D  and/or  3-D  rigid  image  registration  that  is  crucial  to  the 
accurate  change  detection.  We  have  implemented  multi-object  principal-axes  registration  algorithm 
and  are  currently  working  on  the  mutual  information  based  registration  fine-tuning  procedure. 

•  We  have  proposed  a  hybrid  algorithm  for  2-D  and/or  3-D  non-rigid  image  registration,  using  local 
principal-axes  skeleton  mapping  and  surface  based  deformable  warping  with  bilinear  interpolation 
technique. 
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ABSTRACT 

This  paper  presents  an  effective  two-step  scheme  for  automatic  object  detection  in  computed 
radiography  (CR)  images.  First,  various  structure  elements  of  the  morphological  filters,  de¬ 
signed  by  incorporating  available  mophological  features  of  the  objects  of  interets  including 
their  sizes  and  rough  shape  descriptions,  are  used  to  effectively  distinguish  the  foreign  ob¬ 
ject  candidates  from  the  complex  background  structures.  Secondly,  since  the  boundaries  of 
the  objects  are  the  key  features  in  reflecting  object  characteristics,  active  contour  models  are 
employed  to  accurately  outline  the  mophological  shapes  of  the  suspicious  foreign  objects  to 
furhter  reduce  the  rate  of  false  alarms.  The  actual  detection  scheme  is  accomplished  by  jointly 
using  these  two  steps.  The  proposed  methods  are  tested  with  a  database  of  50  hand-wrist 
computed  radiographic  images  containing  various  types  of  foreign  objects.  Our  experimental 
results  demonstrate  that  the  combined  use  of  morphological  filters  and  active  contour  models 
can  provide  an  effective  automatic  detection  of  foreign  objects  in  CR  images  achieving  good 
sensitivity  and  specificity,  and  the  accurate  descriptions  of  the  object  mophological  character¬ 
istics. 

Keywords:  Object  detection,  morphological  filters,  active  contour  models,  computed  ra¬ 

diography. 
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I  Introduction 

Object  detection  and  characterization  through  medical  imaging  technologies  are  among  the 
major  clinically-driven  tasks  towards  the  ultimate  goal  of  diagnosis  and  treatment  of  diseases. 
In  particular,  newly  introduced  computed  radiography  (CR)  has  shown  the  potential  to  re¬ 
place  screen-film  radiography  (SFR)  in  routine  clinical  practice  due  to  its  numerous  advantages 
including  greater  dynamic  range  and  wider  exposure  latitude  [10].  Some  quantitative  compar¬ 
ison  of  CR  and  SFR  in  detection  of  specific  diseases  such  as  pneumothorax  has  demonstrated 
their  equal  detectibility  at  equivalent  exposure  factors  of  two  imaging  systems  [5,  9].  More 
importantly,  due  to  its  native  digital  storage  form,  image  processing  and  analysis  techniques 
can  be  seamlessly  integrated  into  the  CR  system.  The  objective  of  this  work  is  to  develope  an 
automatic  foreign  object  detection  algorithm  which  locates  foreign  objects  in  CR  images  and 
provides  their  accurate  boundary  descriptions  for  further  diagnosis  and  treatment. 

Object  detection  in  CR  images  can  be  accomplished  through  an  accurate  representation  of 
the  object  of  interest  in  terms  of  its  location,  size,  and  shape  description.  In  general,  there  are 
two  general  approaches:  (1)  data-directed  (bottom-up)  approach  such  as  edge  detection  [3]  and 
region  growing  [7];  and  (2)  goal-directed  (top-down)  approach  as  in  knowledge-guided  boundary 
detection  [11].  The  bottom- up  approach  operates  on  the  image  only  based  on  individual  gray 
level  values  to  extract  the  object  boundary  or  to  obtain  the  object  region,  while  the  top-down 
approach  relies  entirely  on  a  priori  constraints  regarding  the  location  and  shape  of  the  object  of 
interest.  In  this  paper,  we  propose  an  automatic  object  detection  technique  which  incorporates 
fundamental  object  characteristics  such  as  the  size  and  generic  shape  information  to  initially 
localize  the  object  of  interest  from  the  scene  image,  and  then  to  confirm  the  correct  presence 
of  suspicious  targets  through  a  boundary  refining  procedure. 

The  presentation  is  organized  as  follows.  In  section  II,  we  present  brief  review  on  object 
detection  problem  and  address  the  difficulties  of  object  detection  in  CR  images.  The  proposed 
new  approache  is  described  in  detail  in  Section  III  where  two  major  steps  (i.e.,  morphological 
filtersa  and  active  contour  models)  are  explained.  In  section  IV,  the  application  of  approach  to 
a  set  of  hand-wrist  CR  images  is  reported  with  the  experimental  results  presented.  Discussion 
and  conclusion  are  provided  in  section  V. 
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II  Problem  Statement 

Automatic  object  detection  is  still  a  difficult  task  in  medical  image  processing  since  many 
clinically  acquired  medical  images  may  be  noisy  and  low  contrast.  In  addition,  the  presence 
of  the  objects  in  question  is  frequently  self-occluded  or  partially  transparent  with  uncertain 
position,  size,  orientation,  shape,  motion,  etc.  An  example  is  shown  in  Figure  1.  Therefore, 
the  precise  boundary  characterization  of  these  objects  may  be  very  difficult. 

Most  research  on  object  detection  falls  into  two  categories:  (1)  edge  detection  and  (2)  region 
growing.  Many  edge  detection  algorithms  have  been  proposed  based  on  the  assumption  that 
different  objects  have  different  intensity  values  [3].  However,  since  most  medical  images  are  low 
contrast  and  noisy,  simple  application  of  edge  detection  will  most  likely  give  broken  boundaries 
of  objects.  On  the  other  hand,  region-based  techniques  often  fail  to  yield  the  desired  structure 
due  to  the  difficulty  of  choosing  a  reasonable  starting  “seed”  point,  an  appropriate  growing 
rule  or  a  suitable  stopping  rule.  More  recently,  knowledge-guided  boundary  finding  methods 
are  proposed  to  extract  geometric  shapes  based  on  site  models  which  are  best  suited  for  objects 
poorly  described  by  features  or  parts.  Boundary  descriptions  using  local  information  fails  to 
give  satisfactory  results  in  practice  because  of  poor-contrast  boundary  regions  due  to  occluding 
and  occluded  objects,  adverse  viewing  conditions  and  noise.  And  further  difficulty  arises  when 
we  try  to  find  a  model  to  describe  a  relatively  broad  class  of  shapes  such  as  that  of  deformable 
objects.  Such  difficulty  in  knowledge  representation  largely  hinders  the  knowledge-guided 
boundary  finding  approach  for  object  detection  problem. 

To  tackle  the  problems  of  forementioned  approaches,  two  important  research  areas  attracted 
our  attention:  active  contour  models  and  morphological  filters.  In  this  paper,  we  propose  to 
combine  morphological  filters  with  active  contour  models  to  automatically  detect  objects  of 
interest  in  CR  images.  The  diagram  of  our  automatic  foreign  object  detection  algorithm  is 
illustrated  in  Figure  1.  Morphological  filters  are  first  applied  to  locate  the  objects  by  con¬ 
centrating  on  location  and  size  information  but  with  a  rough  shape  information.  Then  active 
contour  models  are  used  to  modify  the  contours  of  the  detected  objects.  In  specific,  we  model 
the  initial  contour  extracted  by  morphological  filters  as  a  physical  object,  and  the  data  as  an 
external  force  to  which  the  object  is  attracted,  and  an  iterative  procedure  can  then  be  initiated 
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to  cause  the  active  contour  to  move  toward  the  data  and  ultimately  conform  to  it. 

The  motivation  of  jointly  using  active  contour  models  and  mathematical  mopgological  filters 
can  be  briefly  explained  as  follows.  Active  contour  models,  also  called  snakes,  have  recently 
been  developed  for  finding  optimal  contours  which  offer  the  advantage  that  the  final  form  of 
a  contour  can  be  influenced  by  feedback  from  a  higher-level  process  or  an  interactive  user  [8] . 
The  initial  contour  is  generally  placed  near  the  boundary  of  an  object  under  consideration, 
then  image  forces  draw  the  contour  to  the  object  boundary.  Therefore,  active  contour  models 
are  usually  used  to  interactively  or  semi- automatically  extract  object  boundaries.  On  the  other 
hand,  mathematical  morphology  has  been  an  important  method  for  the  analysis  of  geometric 
structures  of  objects  [2,  6].  It  aims  at  analyzing  the  shape  and  form  of  objects  by  using 
mathematical  set  theory,  topology,  lattice  algebra,  and  random  functions.  As  nonlinear  and 
shape-focused  filters,  morphological  filters  can  suppress  the  background  but  still  retain  size  and 
location  information  with  a  fair  amount  of  accuracy.  The  geometric  nature  of  the  morphological 
filters  is  well  suited  to  perform  object  detection  task. 

Ill  Methods 

In  order  to  achieve  an  automatic  foriegn  onject  detection  in  CR  images,  the  proposed  approache 
consists  of  two  major  steps:  the  first  step  is  to  detect  the  foreign  objects  by  using  morphological 
filters  -  gray-scale  background  reduction  and  binary  opening  operation;  Then  the  active  contour 
models  are  followed  to  refine  the  boundaries  of  those  detected  foreign  objects.  In  this  section, 
we  present  detailed  methametical  formulation  of  these  two  methods  and  describe  our  two-step 
algorithm  in  implementing  their  functions. 

III.l  Foreign  Object  Detection 

Morphological  filters  are  used  to  detect  the  foreign  objects  in  CR  images  whose  background 
is  defined  by  complex  bone  structures,  since  morphological  filters  allow  us  to  suppress  the 
background  while  retaining  size  and  location  information  [2],  [6].  Features  brighter  than  the 
background,  but  smaller  than  the  structuring  element,  can  be  removed  from  an  image  with  the 
opening  operation.  Thus,  if  the  features  of  interest  are  brighter  than  the  background,  opening 
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the  image  by  a  structuring  element  bigger  than  the  largest  feature  will  remove  the  features 
from  the  image  leaving  behind  an  estimate  of  the  background.  Subtracting  the  estimate  of  the 
background  from  the  original  image  extracts  the  features  of  interest.  The  morphological  filter, 
background  reduction ,  is  performed  as  follows  [2]: 

Background  reduction  =  A  —  (A  o  B),  (1) 

where  A  is  the  original  image,  B  is  the  structuring  element  whose  size  is  larger  than  any  of 
the  brighter  features  of  interest,  and  “o”  is  the  gray-scale  opening  operation,  which  is  defined 
as: 


A  o  B  =  (A  ©  B)  ©  B.  (2) 

In  the  above  equation,  ©  and  ©  are  gray-scale  dilation  and  erosion  operations  respectively.  If 
we  denote  the  gray-scale  dilation  image  of  an  image  A  by  a  structuring  element  B  as  C  =  A©B, 
we  can  define  the  element  of  C,  c(x,y),  as 

c(x,y)  =m&x{a(x-i,y-j)  +  b(i,j)\(x-i,y-j)  G  Da;(i,j)  6  Db}.  (3) 

where  a(x,y)  is  the  element  of  A  and  b(x,y )  the  element  of  B;  Da  and  Db  are  the  domains  of 
A  and  B,  respectively. 

Similarly,  an  image  C,  obtained  by  gray-scale  eroding  A  by  B,  denoted  by  C  =  A  ©  B, 
can  be  defined  by 

c(x,y)  =  min {a(x  -  i,y  -  j)  -  b(i,j)\(x  -  i,y  -  j)  €  Da\{i,  j)  €  A,},  (4) 

Note  that  the  location  of  the  extracted  features  by  background  reduction  will  be  exactly 
the  same  as  those  in  the  original  image.  In  gray-scale  morphology,  the  structuring  element  can 
be  any  three-dimensional  structure  such  as  cylinder  or  hemisphere.  In  this  application,  we  use 
n  x  n  hemispherical  mask  whose  individual  element  is  given  by 


w(x,y)  =  yV2  -  ( ~gx/kyt  -  ( ~9VM1 , 


(5) 
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where  w(x,y)  is  the  intensity  at  location  (x,y)  of  the  hemispherical  structuring  element,  g  is 
the  peak  intensity  at  the  center  of  the  mask,  x  and  y  lie  in  the  range  [— fc,  k]  with  k  =  (n—  1) /2. 

Some  small  bright  bony  structures  are  inevitablely  detected  as  foreign  objects  in  hand- 
wrist  CR  images,  but  they  can  be  easily  delineated  by  using  the  size  information  as  their  size 
is  usually  much  smaller  than  that  of  the  foreign  objects  of  interest.  Therefore  we  convert  the 
gray-level  resulting  image  from  the  background  reduction  to  a  binary  image  by  thresholding.  A 
binary  opening  operation  is  then  applied  to  specifically  extract  only  the  foreign  objects.  The 
binary  opening  of  set  A  by  structuring  element  B,  denoted  by  A  o  B,  can  be  defined  as  [6] 

A  o  B  =  (A  0  B)  ©  B,  (6) 

where,  ©  and  ©  denote  binary  dilation  and  erosion ,  respectively.  In  other  words,  equation  (6) 
says  that  the  opening  of  A  by  B  is  simply  the  erosion  of  A  by  B,  followed  by  a  dilation  of  the 
result  by  B.  For  completeness,  we  give  the  definitions  of  binary  dilation  and  erosion  as  follows. 
Let  A  and  B  be  sets  in  Z2,  the  binary  dilation  of  A  by  B,  denoted  by  A  ©  B,  is  defined  as 

A©B  =  {(x!2/)|(B)MnA^0},  (7) 

where  B  =  {(x,y)\x  =  -i,y  =  -j,  for  (i,j)  €  B}  and  (A)(XiJ/)  =  {(c,d)|c  =  i  +  x,d  =  j+y, 
for  (z,  j)  €  A}.  The  binary  erosion  of  A  and  B,  denoted  by  A  ©  B,  is  defined  as 

AQB  =  fel/)|(B)MCA}.  (8) 

which,  in  words,  says  that  the  erosion  of  A  by  B  is  the  set  of  all  points  ( x,y )  such  that  B, 
translated  by  ( x,y ),  is  contained  in  A. 

III. 2  Foreign  Object  Contour  Modification 

The  active  contour  models,  snakes,  are  used  to  modify  the  contours  of  the  foreign  objects 
detected  by  morphological  filters  as  explained  in  the  previous  section.  Starting  from  the  initial 
boundary  of  the  detected  foreign  object,  active  contour  model  uses  data,  gradient  image,  as 
an  external  force  to  cause  the  initial  contour  to  move  toward  the  data  and  ultimately  conform 
to  it.  The  active  contour  model,  snake,  is  defined  as  the  following  mapping  [4]: 
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ft  =[0,1]  -+  5ft2 

s  v(s)  =  (x(s),y(s)).  (9) 

We  define  an  active  contour  model  (snake)  as  a  space  of  admissible  deformations  Ad  and 
a  functional  E  to  minimize.  This  functional  represents  the  energy  of  the  model  and  has  the 
form 


E  :  Ad  -f  $ft 

v  i — ►  E(y)  =  f  teilu^s)!2  +  W2\v"(s)\2  +  P(v(s))ds,  (10) 

Jn 

where  the  primes  denote  differentiation  and  P  the  potential  associated  with  the  external  forces. 
The  mechanical  properties  of  the  model  are  controlled  by  the  functions  Wj.  Their  choice 
determines  the  elasticity  and  rigidity  of  the  model.  If  v  is  a  local  minimum  for  E,  it  satisfies 
the  associated  Euler-Langrange  equation  [4] : 

~{w\v')'  +  {w2v")"  +  \jP{v)  =  0.  (11) 

To  obtain  the  final  solution  of  v,  a  variational  calculus  method  is  originally  proposed  by  Kass 
[8] ,  while  the  dynamic  programming  method  developed  by  Amini  et  al.  allows  addition  of  hard 
constraints  to  obtain  a  more  desirable  behavior  of  the  snakes  [1],  Recently  a  fast  algorithm 
has  been  developed  by  using  a  greedy  algorithm  and  can  be  found  in  [12].  In  this  paper,  we 
use  this  greedy  algorithm  to  modify  the  boundaries  of  foreign  objects  where  the  external  forces 
are  the  gradient  image  data. 

IV  Experimental  Results  and  Discussions 

In  order  to  validate  the  effectiveness  of  the  proposed  method  in  automatic  foriegn  object 
detection  in  CR  images,  intensive  computer  experiments  with  both  simulated  and  real  cases 
have  been  conducted.  In  this  study,  we  have  acquired  three  different  sets  of  CR  images  taken 
from  both  phantom  and  cadaver  hand  specimens.  In  all  these  cases,  the  images  are  embedded 
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with  some  small  foreign  objects  with  different  sizes,  shapes,  and  materials  (like  plastic,  glass, 
graphite,  and  wood).  The  CR  images  are  digitally  acquired  with  a  size  of  1700  x  2000  and 
a  gray-level  resolution  of  lObits/pixel.  An  example  is  shown  in  Figure  2.  Since  in  real-world 
clinical  practice,  CR  images  may  be  taken  with  varying  radiation  dosage  depending  upon  the 
parameter  settings,  and  it  is  desirable  to  obrain  CR  images  with  low  radiation  dosage  while 
maintaining  good  detectibility,  our  testing  database  has  been  designed  to  cover  a  broad  of 
cases  with  by  various  radiation  dosage  [9] . 

The  gray-scale  morphological  filter,  background  reduction ,  is  applied  to  the  original  image 
to  detect  the  foreign  objects  with  different  location,  size,  shape,  and  materials.  The  structuring 
element  is  chosen  as  n  x  n  hemispherical  mask  where  n  (43  in  this  experiment)  corresponds 
to  the  largest  size  among  those  foreign  objects  of  interest.  The  initial  detected  foreign  objects 
are  shown  in  Figure  3  after  application  of  the  gray-scale  morphological  filter  for  background 
reduction.  As  we  can  see  from  Figure  3  all  the  foreign  objects  of  interest  are  detected  without 
any  missing.  This  result  demonstrates  that  morphological  filter  can  effectively  capture  the 
location  and  size  information  of  the  objects  of  interest.  However,  some  redundant  tiny  spots 
are  produced  by  very  bright  bony  structures.  To  delineate  those  small  redundant  spots,  we 
convert  the  gray-level  outcome  from  the  above  step  into  binary  image  simply  by  thresholding, 
(see  Figure  4) .  Then,  we  apply  a  binary  opening  operator  to  explicitly  extract  only  the  foreign 
objects  as  demonstrated  in  Figure  5. 

The  next  step  is  to  refine  the  boundaries  of  those  detected  foreign  objects  by  active  contour 
models.  Taking  the  result  from  morphological  filters  as  initial  contour  (Figure  6)  and  the 
gradient  image  as  the  external  forces,  we  use  the  fast  greedy  algorithm  to  gradually  move  the 
initial  contour  to  the  final  boundary  by  finding  the  minimum  energy  of  the  active  contour 
model.  The  final  boundaries  of  detected  foreign  objects  are  shown  in  Figure  7.  As  we  can  see, 
the  detected  boundaries  conform  very  well  to  those  of  the  foreign  objects  regardless  of  their 
different  locations,  sizes,  shapes,  and  materials. 

However,  our  experience  also  suggests  that  foreign  object  detection  could  be  a  very  diffi¬ 
cult  problem.  For  example,  when  the  acquired  set  of  hand-wrist  CR  images  embedded  with 
some  tiny  invading  objects,  it  is  very  difficult  to  define  the  true  boundaries  of  the  objects  by 
performing  either  edge  detection  or  region  growing,  and  direct  application  of  mophological  fil- 
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tering  maight  be  problematic.  In  other  cases,  when  the  CR  images  are  taken  with  low  dosage, 
the  contrast  will  be  degradated  which  furhter  complicates  the  detection  task  especially  for  the 
complex  bone  structures  that  exist  as  the  background  in  CR  images.  In  this  study,  morpholog¬ 
ical  filters  are  used  to  tackle  the  object  detection  problem  by  focusing  on  location  and  size  of 
the  objects  but  providing  only  rough  shape  information.  Then,  active  contour  models,  which 
are  capable  of  capturing  accurate  shape  information,  are  used  to  refine  the  initially  obtained 
rough  boundaries. 

V  Conclusion 

In  this  paper  we  have  presented  an  automatic  foreign  object  detection  algorithm  by  using 
morphological  filters  and  active  contour  models.  The  morphological  filters  are  used  to  detect 
the  objects  of  interest,  focusing  on  the  location  and  size  information  but  a  rough  shape  infor¬ 
mation,  to  provide  the  initial  contours  to  active  contour  models.  The  active  contour  models 
are  subsequently  employed  to  refine  the  boundaries  of  the  detected  foreign  objects  since  they 
can  successfully  capture  the  shape  information.  The  algorithm  has  been  applied  to  hand-wrist 
CR  images  to  detect  foreign  objects  with  different  locations,  sizes,  shapes,  and  materials.  The 
experimental  results  demonstrate  that  the  new  automatic  foreign  object  detection  algorithm 
provides  the  location  of  the  objects  of  interest  with  their  accurate  boundary  descriptions. 

The  automatic  foreign  object  detection  algorithm  has  been  further  integrated  into  our  task- 
oriented  image  quality  evaluation  method  which  fully  takes  account  the  clinical  purposes  of 
the  medical  images  [14].  The  task-oriented  image  quality  evaluation  method  has  proven  to  be 
very  effective  in  assessment  of  CR  image  quality  for  radiation  dose  optimization  [15].  In  this 
particular  CR  study,  we  can  quantify  the  quality  of  those  images  acquired  with  different  radi¬ 
ation  doses  by  comparing  their  detected  boundaries  of  the  foreign  objects  and  their  segmented 
bone  structures  in  addition  to  the  wavelet  analysis.  The  experimental  results  are  consistent 
with  the  radiologists’  subjective  evaluation  as  reported  in  [9].  In  conclusion,  the  foreign  object 
detection  algorithm  developed  in  this  paper  plays  a  significant  role  in  the  task-oriented  CR 
image  quality  evaluation  technique  to  optimize  radiation  dosage  and  hence  reduce  the  amount 
of  unnecessary  radiation  administered  to  patients  during  diagnostic  procedure. 
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